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(54) PREDICTIVE CODING METHOD AND DECODING METHOD FOR DYNAMIC IMAGE 



(57) Video data predictive encoding and decoding 
methods using the two kinds of prediction modes, the 
global and local motion compensation modes, are pro- 
vided, by which unnecessary MCSEL is reduced as 
much as possible, and the data compression efficiency 
is improved. In the encoder, after a code word MCBPG 
indicating the macroblock type and presence/absence 



of the DOT coefficient of each of two blocks for sending 
color-difference signals, a code word MCSEL indicating 
which motion-compensating mode, global or local, was 
adopted for the prediction of the current macroblock is 
output if the macroblock was not intraframe-encoded. 
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Description 
TECHNICAL FIELD 

5 [0001 ] The present invention relates to methods tor encoding and decoding signals of video data (I.e., moving pic- 
tures). 

BACKGROUND ART 

10 [0002] In existing video data coding standards such as ITU-T H.261, H. 263, ISO/lEC 11172-2 (MPEG-1), and 
ISO/IEC 13818-2 (MPEG-2), a motion-compensated interframe prediction method is adopted for reducing temporal 
redundancy with respect to video data. Also in an example model based on the ISO/IEC 14496-2 (MPEG-4) standard 
which is currently being studied, a similar motion compensating method is adopted. 

[0003] Generally in motion-compensated predictive coding methods, (i) a frame to be encoded (i.e., the current frame) 
15 is divided into rectangular blocks, called "macroblocks", having 16 pixels x 16 lines, (ii) a relative amount of the motion 
(i.e., a motion vector having horizontal component tx and vertical component ty of displacement) with respect to a ref- 
erence frame is detected for each macroblock, and (iii) an interframe difference between a predicted frame and the cur- 
rent frame is encoded, where the predicted frame is obtained in a manner such that the block of the reference frame 
corresponding to the relevant macroblock of the current frame is shifted by the motion vector. 
20 [0004] More specifically, predicted image data (in the reference frame) which most matches the image data at point 
(x, y) of the current frame is represented by using coordinates (x', y") and the above motion vector (t^, ty) as follows. 

x'=x+tx 

25 y'=y+ty 

[0005] That is, the pixel value at the same point (x, y) of the reference frame is not directly used, but the pixel value 
at a point obtained by shifting the point (x, y) by the motion vector (tx, ty) is determined as the predicted value, thereby 
remarkably improving the efficiency of the interframe prediction. 

30 [0006] On the other hand, a global motion compensation method has been proposed, in which motions of the whole 
picture caused by a camera motion such as panning, tilting, or zooming are predicted (refer to H. Jozawa, et al., "Core 
Experiment on Global Motion Compensation (PI) Version 5.0", Description of Core Experiments on Efficient Coding in 
MPEG-4 Video, pp. 1-17. December, 1996). Below, the general structure and operation flow of the encoder and 
decoder used for the global motion compensation will be explained with reference to Figs. 3 and 4. 

35 [0007] First, frame (data) 1 to be encoded (i.e., input frame 1 ) and reference frame (data) 3 are input into global motion 
estimator 4, where global motion parameters 5 relating to the whole frame are determined. Projective transformations, 
bilinear transformations, or aff ine transformations can be used as a motion model in this system. The method disclosed 
by Jozawa et al. can be applied to any motion model so that the kind of motion model is unlimited; however, the general 
functions of the representative motion models as described above will be explained below. 

40 [0008] With any point (x, y) of the current frame and corresponding predicted point (x', y') of the reference frame, the 
projective transformation is represented by the following formula. 

x'= (ax+by-i-t ^) I (px+qy+s) (1 ) 

45 y'= (cx+dy+t y) I (px+qy+s) 

where a, b, c, d, p, q, and s are constants. The projective transformation is a basic form of the two-dimensional trans- 
formation, and generally, the case s=1 in formula (1) is called the projective transformation. If p=q=0 and s=1 , then the 
formula represents the affine transformation. 
so [0009] The following is the formula representing the bilinear transformation. 

x'= gxy+ax+by+tjj (2) 

y'= hxy+cx+dy+ty 

55 

where a, b, c, d, g, and b are constants. If g=h=0 in this formula, then the affine transformation can also be obtained 
as the following formula (3). 
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x'=ax+by+tx (3) 
y'=cx+dy+ty 

5 [001 0] In the above formulas, tx and ty respectively represent the amounts of parallel shifting motions in the horizontal 
and vertical directions. Parameter "a" represents an extension/contraction or inversion effect in the horizontal direction, 
while parameter "d" represents an extension/contraction or inversion effect in the vertical direction. Parameter "b" rep- 
resents a shearing effect in the horizontal direction, while parameter "c" represents a shearing effect in the vertical 
direction. In addition, the condition that a=cose , b=sine , c=-sine , and d=cose represents rotation by angle 9. The 

10 condition that a=d=1 and b=c=0 represents a model equal to a conventional parallel motion model. 

[001 1 ] As explained above, the motion model employing the aff ine transformation can represent various motions such 
as parallel shift, extension/contraction, inversion, shear and rotation and any composite motions consisting of a few 
kinds of the above motions. Projective or bilinear transformations having many more parameters can represent more 
complicated motions. 

15 [001 2] The global motion parameters 5 determined in the global motion estimator 4 are input into global motion com- 
pensated predictor 6 together with reference frame 3 stored in frame memory 2. The global motion compensated pre- 
dictor 6 makes the motion vector (for each pixel) calculated using the global motion parameters 5 act on the reference 
frame 3, so as to generate global motion-compensating predicted frame (data) 7. 

[0013] On the other hand, the reference frame 3 stored in the frame memory 2 is input into local motion estimator 8 
20 together with input frame 1 . In the local motion estimator 8, motion vector 9 between the input frame 1 and the reference 
frame 3 is detected for each macroblock of 16 pixels x 16 lines, In the local motion compensated predictor 10, local 
motion-compensating predicted frame (data) 1 1 is generated using the motion vector 9 of each macroblock and the ref- 
erence frame 3. The above operation corresponds to the conventional motion compensation method used in MPEG or 
the like. 

25 [0014] Next, the prediction mode determining section 12 chooses one of the global motion-compensating predicted 
frame 7 and the local motion-compensating predicted frame 1 1 for each macroblocK the chosen one having a smaller 
error with respect to the input frame 1 . The predicted frame 13 chosen by the prediction mode determining section 12 
is input into subtracter 1 4, and a difference frame 1 5 between the input frame 1 and the predicted frame 1 3 is converted 
into DOT coefficients 17 in DOT (discrete cosine transform) section 16. Each DOT coefficient 17 obtained by the DOT 

30 section 16 is further converted into quantized index 19 in quantizer 18. The quantized index 19, global motion parame- 
ters 5, motion vector 9, and prediction mode information 26 showing the determined prediction mode output from the 
prediction mode determining section 12 are respectively encoded in encoding sections 101 to 104, and then multi- 
plexed in the multiplexer 27' so as to generate encoder output (i.e., encoded bit sequence) 28'. 
[0015] In order to make the reference frames in both the encoder and decoder agree with each other, the quantized 

35 index 19 is restored to quantization representative value 21 by inverse quantizer 20. and then inversely converted into 
difference frame 23 by inverse DCT section 22. The difference frame 23 and the predicted frame 1 3 are added in adder 
24, so that locally decoded frame 25 is obtained. This locally decoded frame 25 is stored in frame memory 2, and is 
used as a reference frame when the next frame is encoded. 

[001 6] In the decoder (see Fig. 4), the encoded bit sequence 28' which was received is separated using demultiplexer 
40 29' into lour encoded components, that is, quantized index 1 9, prediction mode information 26, motion vector 9, and glo- 
bal motion parameters 5. These four components are respectively decoded by decoding sections 201 to 204. The ref- 
erence frame 3 (equal to the reference frame 3 as shown in Fig. 3) stored in frame memory 33 is input into global motion 
compensated predictor 34 together with the decoded global motion parameters 5. The global motion compensated pre- 
dictor 34 makes the global motion parameters 5 act on the reference frame 3 so as to generate global motion-compen- 
45 sating predicted frame 7 which is the same as frame 7 in Fig. 3, The reference frame 3 is also input into local motion 
compensated predictor 35. In the local motion compensated predictor 35, the motion vector 9 acts on the reference 
frame 3 so as to generate local motion-compensating predicted frame 1 1 which is also the same as frame 1 1 in Fig. 3. 
[0017] In the following step, the global and local motion-compensating predicted frames 7 and 1 1 are input into pre- 
diction mode determining section 36. In the prediction mode determining section 36, one of the global and local motion- 
50 compensating predicted frames 7 and 1 1 is chosen based on the decoded prediction mode information 26. The chosen 
frame is determined as predicted frame 13. 

[0018] The decoded quantized index 19 is restored to quantization representative value 21 in inverse quantizer 30, 
and then inversely converted into difference frame 23 in the inverse DCT section 31. The difference frame 13 and the 
predicted frame 23 are added in adder 32 so that locally decoded frame 25 is obtained, This locally decoded frame 25 
55 is stored in frame memory 33 and is used as a reference frame when the next frame is decoded. 

[0019] In the global motion -compensated prediction method in the above-explained conventional technique, one of 
the predicted images, which has the smaller prediction error, obtained by the global and local compensated methods is 
chosen for each macroblock so as to improve the prediction efficiency over the whole frame. To implement such a sys- 
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tem, it is necessary to insert a code word in the encoded data sequence, which represents which prediction method 
(among the global motion compensation and the local compensation] was used. This is because the decoder must be 
informed of which motion compensating method was used for the prediction of each macroblock. Therefore, in a pro- 
posal (by the present inventors) for the MPEG-4 currently examined for standardization, the encoded data structure 
5 (i.e., syntax) of the macroblock is as that shown in the following List 1 . In List 1 , the encoded data sequence is described 
using pseudo-C codes, and operations of the encoder and decoder are also described. Fig. 5 is a model diagram show- 
ing the data structure (i.e., bit stream structure) represented by List 1 , in which data are constructed using code words 
D1 to D8. the motion vector, and DOT coefficient information (corresponding to the quantized index) in turn. 



List I 



15 



SYNTAX 



NUMBER 
OF BITS 



56 



macroblock( ) { 

if (VOP.iypc mm "SPRITE") 

MCSEL 
if(VOP_iypc — "n { 
(OMITTED) 

> 

if (VOF.iypc ~ "P" II VOP.iype — "SPRITE") < 
COD 

if (COD — 1) 

rctum( ) 

MCBPC 

if (MBTYPE — "INTRA- H MBTYPE — 'TNTRA+Q") 
Acpred^flag 

CBPY 

if (MBTYPE — "INTER+Q" N MBTYPE — "INTRA-MJ'*) 
DQUANT 

if (MBTYPE !- "INTRA" MBTYPE !- "INTRA-KJ") < 
if (VOP^typc — -SPRITE") { 
if (MCSEL — 0) 

molioii_vcctor(MBTYPE) 

> 

else{ 

iiioaon,vector(MBTYPE) 

> 

} 

for (i -0; i < block_count; i-M-) 
block ( ) 

> 

else if (VOP.typc — "B*0 < 
(OMITTED) 

> 



1-9 
1 

1-Z 
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[0020] In MPEG-4, a conventional frame is called VOP (video object plane). The VOP has lour types as shown in the 
following List 2. 

5 



List 2 


VOP Type 


Prediction Method 


1 


Intraframe Coding 


P 


Interlrame (Predictive) Coding 


B 


Bi-directional Interframe Coding 


SPRITE 


Sprite Coding (using the background of the whole sequence or the global motion compensation) 



[0021] The P-, and B-VOPs are the same as I-, P-, and B-pictures defined in MPEG-1 or MPEG-2. The SPRITE- 
VOP is a newly introduced concept in l\/lPEG-4, in which prediction is performed based on the background picture over 
the whole part of a video clip in a video data sequence (such a background image being called the "static sprite") or on 
20 the "dynamic sprite" obtained by the global motion compensation. In the syntax shown in List 1 , descriptions relating to 
the l-VOP and B-VOP are omitted for simplifying the explanations. Additionally, in MPEG-4, a video object of any form 
can be encoded and thus, shape information is also described in the relevant syntax; however, such shape information 
is also omitted for simplifying the explanations. 

[0022] In a global motion-compensated predictive encoder suitable for the syntax of List 1 , if the VOP type is SPRITE, 
25 then a 1-bit code word "MCSEL" (see reference symbol D1 in Fig. 5) is output as the prediction mode information 26. 
MCSEL is a flag indicating which of the global motion compensation and the local motion compensation was used for 
the prediction of the current macroblock. If the global motion compensation was used, then MCSEL=1 , while if the local 
motion compensation was used, then MCSEL=0. 

[0023] If the VOP type is P or SPRITE, then a 1-bit code word "COD" (see reference symbol D2) is output. COD is a 
30 flag indicating whether the current macroblock was skipped. If no skip and encoding was performed, then COD=0, while 

if the macroblock was skipped, then C0D=1 . The skipping of the macroblock occurs when the type of the macroblock 

is INTER, the motion vector is (0,0), and all DOT coefficient values are zero. In this case, it is unnecessary to encode 

the macroblock type, information of the motion vector, and the DCT coefficient: thus, a large compression is possible. 

If COD=0, then the operation proceeds to the next step, while if C0D=1 , then all the following steps (relating to the cur- 
35 rent macroblock) are skipped and the operation necessary Ibr processing the next macroblock is started. 

[0024] In the next step, the encoder outputs a variable-length code word "MCBPC" (see reference symbol D3). 

MCBPC indicates the macroblock type and the absence/presence of the DCT coefficient of each of two blocks which 

are selected for sending color-difference signals. 

[0025] The macroblock has the following five types (or modes) shown in the following List 3. 

40 



Ust3 


Macroblock Type 


Prediction Method 


INTER 


Interframe Prediction 


INTER+Q 


Interframe Prediction (using quantization step information) 


INTER4V 


Interframe Prediction (using four motion vectors) 


INTRA 


Intraframe Prediction 


INTRA+Q 


Intraframe Prediction (using quantization step information) 



55 [0026] If the macroblock type belongs to the intraframe coding mode, that is, is INTRA or INTRA+Q, then code word 
"AcpredJ lag" (see reference symbol D4) is output. "AcpredJIag" is a flag indicating whether the AC (alternating cur- 
rent) coefficient prediction of the DCT was performed with respect to the current macroblock. If the AC coefficient pre- 
diction was performed, then Acpred_flag=1, while if no AC coefficient prediction was performed, then Acpred_flag=0. 
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[0027] The encoder then outputs code word "CBPY" (see reference symbol D5). CBPY indicates whether the DCT 
coefficients were determined with respect to four blocks for sending brightness signals. If the macrobiock type is 
INTER+Q or INTRA+Q, then quantization step information DQUANT (variable-length code word: D6) is output. 
[0028] Next, if the macrobiock type does not belong to the intraframe coding mode, that is, is neither INTRA nor 

5 INTRA+Q, then motion vector information (see reference symbol D7) is output. Here, if the VOP type is SPRITE, then 
the motion vector information (D7) is output only when MCSEL=0, that is, when the local motion compensation was 
employed, and thus no motion vector information is output when the global motion compensation is employed. 
[0029] In the last step, the DCT coefficient information of each 8x8 block included in the 16x1 6 macrobiock is output 
as quantized index 19 (see reference symbol D8). 

10 [0030] In the above-explained syntax, if the macrobiock type belongs to the intraframe coding mode (such as INTRA 
and INTRA+Q). then MCSEL is output also in this case. In the intraframe coding mode, neither global nor local motion 
compensation is performed; thus, a decision of MCSEL is useless. Therefore, in this case, there occurs the problem that 
1 -bit of unnecessary data is added for each macrobiock. 

[0031 ] In addition, if the global motion compensation is effective (for a frame to be encoded), the macrobiock skipping 
15 is generally performed in the global motion compensation mode, and the macrobiock skipping is rarely performed in the 

local motion compensation mode. TTierefore, also in the case of the macrobiock skipping, MCSEL is practically useless 

and there also occurs the problem that 1 -bit of unnecessary data is added for each macrobiock. 

[0032] If the transmission rate is high, such overhead data occupies a very small portion of the whole data; thus, no 

serious problem occurs. However, as the Internet has become widespread very rapidly, video data transmission with a 
20 low transmission rate has been required recently. In the encoding of video data having a low-transmission rate, the rate 

of overhead data to the whole data is inevitably increased. Therefore, the necessity of reducing such overhead data has 

also increased. 

[0033] More specifically, the code word MCSEL takes only one bit per macrobiock. However, in a CIF (common inter- 
face format) picture of 352 pixels x 288 lines, MCSEL occupies 396 bits per frame, while in a QCIF (quarter common 
25 interface format) picture of 176 pixels x 144 lines, MCSEL occupies 99 bits per frame. The amount of MCSEL is fixed 
regardless of the encoding rate; thus, in the low-rate encoding, the amount of MCSEL increases and it may be a great 
burden on the system. For example, if QCIF pictures with a transmission rate of 1 0 frames /sec are encoded with a 20 
kbit/sec, then MCSEL occupies a data amount of 99x1 0»1 kbit/sec, which thus occupies almost 5% of the whole data 
rate. 

30 

DISCLOSURE OF THE INVENTION 

[0034] The inventors of the present invention noted the above-described requirement at the start, and tried to solve 
the above-explained problems. That is, the present invention relates to a video data (i.e., moving pictures) predictive 
35 coding method using two kinds of prediction modes, the global and local motion compensation modes, and the objec- 
tive thereof is to provide a video data predictive encoding method and a corresponding decoding method for reducing 
unnecessary MCSEL as much as possible, and improving the data compression efficiency 

[0035] To achieve the above objective, the present invention provides a predictive encoding method of video data, in 
which one of a global motion-compensating process for predicting a global motion of the whole frame and a local 
40 motion-compensating process for predicting a local motion of each block In a frame is selectively performed, wherein: 

if a current block to be processed was interframe-encoded, then a code word for indicating the prediction mode is 
inserted in an encoded data sequence of the current block, the code word indicating which of the global and local 
motion-compensating processes was used for predicting the current blocK and the code word inserted after 
45 another code word indicating the encoding mode of the current block; 

othenwise, the code word for indicating the prediction mode is not inserted in the data sequence. 

[0036] The above is the first method. 

[0037] In the above method, it is possible that when the current block is block-skipped, the global motion-compensat- 
50 ing process is always chosen and in the skipped block, the code word for indicating the prediction mode is omitted. This 
is the second method of the present invention. 

[0038] The present invention also provides a decoding method for decoding a data sequence encoded by the above 
first method, wherein: 

55 if the current block was interframe-encoded, then the code word for indicating the prediction mode is extracted from 
the data sequence and decoding is performed using the indicated prediction method; 
otherwise the code word for indicating the prediction mode is not extracted. 
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[0039] The present invention also provides a decoding method for decoding a data sequence encoded by the above 
second method, wherein when the current block has been block-skipped, the code word Ibr indicating the prediction 
mode is not extracted and a decoding process corresponding to the global motion-compensating process is performed. 
[0040] As described above, if the macroblock type belongs to the intraframe coding mode, that is, the type is INTRA 

5 or INTRA+Q, then neither the global motion compensation method nor the local motion compensation method is used; 
thus, a flag (MCSEL) for indicating which method was adopted is unnecessary. However, in the conventional methods, 
IVICSEL is positioned before the code word (MCBPC) for indicating the macroblock type: therefore, the decoder cannot 
determine whether MCSEL is necessary until MCBPC is extracted in the decider. In this case, regardless of whether 
the macroblock type is the intraframe coding mode, MCSEL must be added to every macroblock. 

10 [0041 ] In comparison, according to the above first method according to the present invention, MCSEL is inserted after 
MCBPC; thus, after the decoder reads out the macroblock type, the decoder can determine whether MCSEL appears. 
Therefore, in the intraframe coding mode, it is unnecessary to add MCSEL, thereby reducing overhead data. 
[0042] Also as explained above, if the global motion compensation is effective (for a frame to be encoded), the mac- 
roblock skipping is generally performed in the global motion compensation mode, and the macroblock skipping is rarely 

15 performed in the local motion compensation mode. Therefore, also in the case of the macroblock skipping, MCSEL is 
practically useless. 

[0043] According to the above second method, the macroblock skipping can be limitedly performed in the global 
motion compensation, thereby omitting MCSEL at the macroblock skipping and further reducing unnecessary overhead 
data. 

20 [0044] That is, according to the predictive encoding and decoding methods of video data of the present invention, 
unnecessary MCSEL data can be reduced as much as possible, so that overhead data can be reduced and the data- 
compression efficiency can be improved, The lower the encoding rate, the clearer the effect of the present invention. 
[0045] The present invention also provides a storage medium storing a program for making a computer execute any 
method as described above, and a storage medium storing data encoded by any encoding method as described above. 

25 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0046] 

30 Fig. 1 is a block diagram showing the structure of the encoder used in an embodiment according to the present 
invention. 

Fig. 2 is a block diagram showing the structure of the decoder used together with the encoder of Fig. 1 . 

Fig. 3 is a block diagram showing the structure of a conventional example of the encoder. 

Fig. 4 is a block diagram showing the structure of the decoder used together with the encoder of Fig. 3. 
35 Fig. 5 is a model diagram showing the data structure (i.e., bit stream structure) represented by List 1 . 

Fig. 6 is a model diagram showing the data structure (i.e., bit stream structure) represented by List 4. 

Fig. 7 is a model diagram showing the data structure (i.e., bit stream structure) represented by List 5. 

Fig. 8 is a flowchart (1) corresponding to the whole prediction encoding program of video data. 

Fig 9 is a flowchart (2) corresponding to the whole prediction encoding program of video data. 
40 Fig. 10 is a flowchart (1) showing a conventional example of the macroblock layer multiplexing operation. 

Fig. 11 is a flowchart (2) showing a conventional example of the macroblock layer multiplexing operation. 

Fig. 12 is a flowchart (1) showing an example of the macroblock layer multiplexing operation according to the (first) 

method of the present invention. 

Fig. 13 is a flowchart (2) showing an example of the macroblock layer multiplexing operation according to the (first] 
45 method of the present invention. 

Fig. 1 4 is a flowchart (1) showing an example of the macroblock layer multiplexing operation according to the (sec- 
ond) method of the present invention. 

Fig. 15 is a flowchart (2) showing an example of the macroblock layer multiplexing operation according to the (sec- 
ond) method of the present invention. 
50 Fig. 1 6 is a flowchart corresponding to the whole prediction decoding program of video data. 

Fig. 17 is a flowchart (1) showing a conventional example of the macroblock layer demultiplexing and decoding 
operation. 

Fig. 18 is a flowchart (2) showing a conventional example of the macroblock layer demultiplexing and decoding 
operation. 

55 Fig. 19 is a flowchart (1) showing an example of the demultiplexing and decoding operation for a macroblock layer 
multiplexed using the above first method. 

Fig. 20 is a flowchart (2) showing an example of the demultiplexing and decoding operation for a macroblock layer 
multiplexed using the above first method. 
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Fig. 21 is a flowchart (1) showing an example of the demultiplexing and decoding operation for a macroblocl^ layer 
multiplexed using the above second method. 

Fig. 22 is a flowchart (2) showing an example of the demultiplexing and decoding operation for a macroblock layer 

multiplexed using the above second method. 

5 

MODES FOR CARRYING OUT THE INVENTION 

[0047] Hereinbelow, preferred embodiments of the present invention will be explained with reference to the drawings. 
[0048] Fig. 1 is a block diagram showing the structure of the encoder used in an embodiment according to the present 

10 invention, In comparison with Fig. 3 showing a conventional encoder, only the operation of multiplexer 27 is different in 
this encoder. This is because the basic structural elements in the encoder and decoder and operation flows according 
to the present invention are the same as those in the conventional system. However, the structure of the encoded data 
sequence is different; thus, the encoded data sequence (i.e., syntax) according to the present invention will be 
explained below with reference to List 4. Fig. 6 is a model diagram showing the data structure (i.e., bit stream structure) 

15 represented by List 4, in which data are constructed using code words D1 1 to D19, the motion vector, and DOT coeffi- 
cient information (con^esponding to the quantized index) in turn. 
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list 4 



SYNTAX 



NUMBER 
OF BITS 



macroblcKk( ) { 

if (VOPjype —"I") { 
(OMITTED) 

> 

if (VOP^iypc — "P" II VOP.typc — "SPRITE'') { 
COD 

if(COD~ 1) { 

if (VOP_lypc — "SPRITE") 
MCSEL 

retumC ) 

} 



"INTER" • MBTYPE — "INTER^Q")) 



MCBPC 

tf ( VOP.typc — "SPRITE" 
&& (MBTYPE - 
MCSEL 

if (MBTYPE — "INTRA" II MBTYPE — "INTRA+QT 
CBPY 

if (MBTYPE ~ "INTER+Q" II MBTYPE — "INTRA+Q") 
DQUANT 

if (MBTYPE !- "INTRA" && MBTYPE !- INTRA+Q'*) < 
if (VOP^typc — "SPRITE") { 
if (MCSEL — 0) 

motion_vectar(MBTYPE) 

> 

else { 

raoUon.vector(MBTYPE) 

> 

> 

for (i -0; i < block.count; 
block ( ) 

> 

else if (VOP_type — "B") < 
(OMITTED) 

> 



1-9 



1 

2-6 
1-2 



50 



[0049] As shown in List 4, in the present embodiment, if the VOP type is P or SPRITE, then the above-explained 1- 
bit code word COD (see reference symbol 011} Is output. The macroblock skipping is performed if the macroblock type 
is INTER, the motion vector is (0, 0), and all DOT coefficients are zero, as in the conventional method. However, infor- 
55 mation indicating the adopted kind of motion compensation (that is, global or local) is necessary here. Therefore, even 
when C0D=1 , if the VOP type is SPRITE, then MCSEL (i.e., (determined) prediction mode information) 26 (see refer- 
ence symbol D12) is output on this particular condition. Here, if the global motion compensation is used, then 
MCSEL=1 , while if the local motion compensation is used, then MCSEL=0. 
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[0050] If C0D=1 , then all the following steps (relating to the current macroblock) are skipped and the operation nec- 
essary for processing the next macroblock Is started. In this case, it is unnecessary to encode the macroblock type, the 
motion vector information 9, and the DCT coefficient; thus, a large compression is possible. 
[0051 ] In the next step, the encoder outputs a variable-length code word MCBPC (see reference symbol D1 3). 
5 [0052] Next, if the VOP type is SP RITE and the macroblock type is INTER or INTE R+Q, the above mentioned MCSEL 
26 (see reference symbol D14) is output after the above MCBPC. 

[0053] The choosing operation between the global and local motion compensation modes Is performed in the predic- 
tion mode determining section 12, as in the conventional system. Any determination method can be adopted, for exam- 
ple, a method for reducing a prediction error, or another method in which both the prediction error and the amount of 
10 generated codes are considered. 

[0054] Next, if the macroblock type belongs to the intraframe coding mode, that is, the type is INTRA or INTRA+Q, 
then the above-explained code word "AcpredJIag" (see reference symbol D15) is output. 

[0055] The encoder then outputs the above-explained code word CBPY (see reference symbol D1 6). In the next step, 
if the macroblock type is INTER-i-Q or INTRA+Q, then the above-explained quantization step information DQUANT (var- 
15 iable-length code word: see reference symbol D1 7) is output. Next, if the macroblock type does not belong to the intra- 
frame coding, that is, the type is neither INTRA nor INTRA+Q, then motion vector 9 (see reference symbol D18) is 
output. Here, when the VOP type is SPRITE, if MCSEL=1 , that is, if the global motion compensation was chosen, then 
the motion vector 9 is not output, and only if MCSEL=0, that is. the local motion compensation was chosen, then the 
motion vector 9 is output. 

20 [0056] Lastly, the DCT coefficient information of each 8x8 block included in the 1 6x1 6 macroblock is output as quan- 
tized index 19 (see reference symbol D19). 

[0057] The quantized index 1 9 of the DOT coefficient, prediction mode information (MCSEL) 26, motion vector 9, and 
global motion parameters 5 are respectively encoded in encoding sections 101 to 104, and then multiplexed in the mul- 
tiplexer 27 according to the above-described operation so as to generate encoder output 28. Here, each VOP needs a 

25 set of global motion parameters 5, thus, is not included in the syntax (i.e.. List 4) relating to the macroblock layer. Also 
in the syntax of List 4, descriptions relating to the l-VOP and B-VOP are omitted for simplifying the explanations. Addi- 
tionally in MPEG-4, a video object of any form can be encoded and thus, shape information is also described in the 
relevant syntax, as explained above. However, such shape information is also omitted for simplifying the explanations. 
[0058] Fig. 2 is a block diagram showing the structure of the decoder used together with the encoder of Fig. 1 . In com- 

30 parison with the structure of the conventional decoder as shown in Fig. 4, only the operation of demultiplexer 29 is dif- 
ferent. 

[0059] Below, the operation of the present decoder will be explained with reference to Fig. 2, List 4, and Fig. 6. 
[0060] As shown in List 4. in the decoder of the present embodiment, first, if the VOP type is P or SPRITE, then code 
word COD (D11) is extracted (or read out). Next, if C0D=1 and the VOP type is SPRITE, then MCSEL (D12) is 
35 extracted. 

[0061] The code word MCSEL activates selector 36, and if MCSEL=1, then global motion-compensating predicted 
frame 7 is chosen as predicted frame 1 3. If MGSEL=0, all DOT coefficients are zero; thus, the acquired predicted frame 
13 itself is output as decoded frame 25 of the current macroblock. 

[0062] If GOD=0. then the variable-length code word MCBPC (D13) is extracted. According to the extracted MCBP, 
40 information on the macroblock type and the absence/presence of the DCT coefficient of each of two blocks selected for 
sending color-difference signals is obtained. 

[0063] Following to MCBPC, if the VOP type is SPRITE and the macroblock type is INTER or INTER+Q, then MCSEL 
(D14) is extracted. 

[0064] Next, if the macroblock type is INTRA or INTRA+Q, then the code word AcpredJ lag (D1 5) is extacted. 
45 [0065] Then, CBPY (D16) is extracted so as to obtain decoded information on presence/absence of the DCT coeffi- 
cients with respect to brightness signals. In the next step, if the macroblock type is INTER+Q or INTRA+Q, then the var- 
iable-length code word, quantization step information DQUANT (D17), is extracted. 

[0066] Next, if the macroblock type is neither INTRA nor INTRA+Q, motion vector 9 (D18) is extracted. Here, when 
the VOP type is SPRITE, if MCSEL^I, that is, if the global motion compensation was adopted, then no motion vector 
so information exists and thus motion vector 9 is not extracted, while if MCSEL=0, that is, the local motion compensation 
was adopted, then motion vector 9 is extracted on this particular condition. MCSEL activates the selector 36. If 
MCSEL=1. then the global motion-compensating predicted frame 7, acquired using reference frame 3 and global 
motion parameters 5, is determined as predicted frame 13, while if MCSEL=0, then the local motion-compensating pre- 
dicted frame 1 1 , acquired using reference frame 3 and global motion parameters 9, is determined as predicted frame 

55 13. 

[0067] In the next step, DCT coefficient information (D1 9) on each 8x8 block included in the relevant macroblock, that 
is, quantized index 19, is extracted. The quantized index 19 is restored to quantization representative value 21 by 
inverse quantizer 30, and then inversely converted into difference frame 23 by inverse DCT section 31 . The difference 
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frame 23 and the predicted frame 13 are added in adder 32, so that decoded frame 25 is obtained. This decoded frame 
25 is stored in frame memory 33, and is used as a reference frame when the next frame is decoded. 
[0068] Hereinbelow, a predictive coding method for reducing overhead data in the macroblock stripping will be 
explained with reference to Fig. 1 and the following List 5. Fig. 7 is a model diagram showing the data structure (i.e., bit 
5 stream structure) represented by List 5, in which data are constructed using code words D21 to D28, the motion vector, 
and DCT coefficient information in turn. 
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Lists 



15 



20 



25 



30 



35 



SYNTAX 

macroblockC ) { 

if(VOPjype — T'X 
(OMITTED) 

> 

if (VOP^type — T'' » VOP^lype — "SPRITE") < 
COD 

if (COD — 1) 

retuni( ) 

MCBPC 

if (VOP_typc — "SPRITE" && 

(MBTYPE — -INTER" M MBTYPE — "INTER-KJ-) 
MCSEL 

if (MBTYPE — '"INTRA" tt MBTYPE — "INTRA*Q") 
Acpred^nag 

cBpy 

if (MBTYPE — "INTER-KJ" II MBTYPE — -INTRA+Q*') 
DQUANT 



NUMBER 
OF BITS 



1-9 



1 
2*6 

1-2 



40 



45 



50 



55 
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if (MBTYPE !- "INTRA- AA MBTYPE !- "INTRA^") ( 
if (VOP .type — "SPRITE") { 
if(MCSEL — 0) . 

nu>tioiuvector(MBTYPE) 

) 

eUe< 

nioiion_vector(MBTyPE) 

) 

> 

for (i -O; i < block.counc; i-t-f ) 
block ( ) 

> 

else if (VOP^ype— -B") { 
(OMITTED) 

> 



[0069] In List 5, if the VOP type is P or SPRITE, then code word COD (see reference symbol D21) is output. The mac- 
25 roblock skipping indicated by COD is adopted when the global motion condensation is used and all DCT coefficients 
are zero, that is, the macroblock skipping is not performed when the local motion compensation Is adopted. Therefore, 
relevant IVICSEL appearing In List 4 Is omitted In List 5. In the macroblock skipping, all the following steps with respect 
to the current macroblock are skipped and the operation for processing the next macroblock is started. Therefore, it is 
unnecessary to encode the macroblock type and the DCT coefficient of the current macroblock; thus, a large compres- 
30 sion can be achieved. 

[0070] Next, the encoder outputs the variable-length code word MCBPC (see reference symbol D22). 
[0071] Followling to IVICBPC, If the VOP type is SPRITE, and the macroblock Is INTER or INTER+Q, MCSEL (see 
reference symbol D23) Is output. Any method can be adopted for determining the motion-compensating mode by the 
prediction mode determining section 12, for example, a method for reducing a prediction error, or another method in 
35 which both the prediction error and the amount of generated codes are considered. 

[0072] Next, if the macroblock type is the intraframe coding mode, that Is, INTRA or INTRA+Q, then code word 
AcpredJ lag (see reference symbol D24) is output. 

[0073] In the next step, the encoder outputs the variable-length code word CBPY (see reference symbol D25). Next, 
if the macroblock type is INTER+Q or INTRA+Q, then the variable-length code word, quantization step information 

40 DQUANT (see reference symbol D26), is output. 

[0074] Next, If the macroblock type does not belong to the intraframe coding mode, that is. Is neither INTRA nor 
INTRA+Q, then motion vector 9 (see reference symbol D27) Is output. Here, when the VOP type Is SPRITE, if 
MCSEL=1 , that Is, If the global motion compensation was adopted, then no motion vector information Is output, while If 
MCSEL=0, that Is, if the local motion compensation was adopted, then the motion vector is output on this particular con- 

45 ditlon. 

[0075] Lastly, DCT coefficient information 9 (see reference symbol D28} with respect to each 8x8 block Included in 
the current macroblock is output. 

[0076] The multiplexer 27 then multiplexes quantized Index 19, prediction mode information (MCSEL] 26, motion vec- 
tor 9, and global motion parameters 5 (which were respectively output as explained above), and outputs an encoder 

50 output 28. Here, the global motion parameters 5 are not Included in the syntax of List 5, as In List 4. 

[0077] In the syntax of List 5, descriptions corresponding to l-VOP and B-VOP are omitted for simplicity. Additionally, 
the above-described shape information with respect to MPEG-4 is also omitted in the relevant syntax. 
[0078] Below, the operation of the decoder for decoding data encoded by the above-explained predictive encoding 
method will be explained with reference to Fig. 2, List 5, and Fig. 7. 

55 [0079] In the decoding method shown in List 5, first, if the VOP type is P or SPRITE, then the code word COD (see 
reference symbol D21) Is extracted. If C0D=1, then the global motion-compensating predicted frame 7, obtained using 
reference frame 3 and global motion parameters 5, is determined as predicted frame 13. If C0D=1, all the DCT coeffi- 
cients are zero; thus, the obtained predicted frame 13 Itself is output as the decoded frame 25 of the current macrob- 
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lock. 

[0080] If COD=0, then the variable-length code word MCBPC (see reference symbol D22) is extracted so as to obtain 
decoded information on the macroblock type and the presence/absence of DOT coefficients with respect to two signals 
indicating color difference. 

5 [0081 ] Following to MCBPC, if the VOP type is SPRITE and the macroblock type is INTER or INTER+Q, then MCSEL 
(see reference symbol D23) is ©ctracted. 

[0082] Next. If the macroblock type is INTRA or INTRA+Q, then code word AcpredJ lag (see reference symbol D24) 
is extracted. 

[0083] In the next step, CBPY (see reference symbol D25) is extracted so as to obtain decoded information on the 
10 DOT coefficients with respect to the brightness signals. Next, if the macroblock type is INTER+Q or INTRA+Q, then 
quantization step information DQUANT (see reference symbol D26), a variable-length code word, is extracted. 
[0084] Next, if the macroblock type is neither INTRA nor INTRA+Q, then motion vector 9 (see reference symbol D27) 
is extracted. Here, when the VOP type is SPRITE, if MCSEL=1, that is, the giobai motion compensation was used, no 
motion vector exists and thus the motion vector extraction is not performed. If MCSEL=0, that is, the local motion com- 
15 pensation was used, motion vector 9 is extracted on this particular condition. MCSEL activates selector 36, and if 
MCSEL=1 , then global motion-compensating predicted frame 7, obtained using reference frame 3 and global motion 
parameters 5, is determined as predicted frame 13. If MCSEL=0, then local motion-compensating predicted frame 1 1 , 
obtained using reference frame 3 and motion vector 9, is determined as predicted frame 13. 

[0085] Next, DCT coefficient information (see reference symbol D28) on each 8x8 block included in the current mac- 
20 roblock, that is, quantized index 19 is extracted. The quantized index 19 is restored to quantization representative value 
21 in inverse quantizer 30, and then inversely converted into difference frame 23 in the inverse DCT section 31. The 
difference frame 13 and the predicted frame 23 are added in adder 32 so that decoded frame 25 is obtained, This 
decoded frame 25 is stored in frame memory 33 and is used as a reference frame when the next frame is decoded. 
[0086] Flowcharts of programs for executing the above-explained methods in the above embodiments of the present 
25 invention are appended. In addition, the flowcharts for executing the conventional methods are also appended for com- 
parison. 

[0087] In the flowcharts, some steps belonging to the conventional process such as the decision of the VOP type and 
the decision or determination of the macroblock type (in the determination, a most efficient macroblock type is selected) 
are omitted. These omitted steps can be considered unnecessary for explaining the present invention. 
30 [0068] Figs. 8 and 9 are flowcharts corresponding to the whole predictive encoding program of video data (i.e., moving 
pictures). In the figures, "(D" in Fig. 8 indicates that this line continues to "(D" in Fig. 9, while "® " in Fig. 8 continues to 
"@" in Fig. 9. The multiplexing process in the double-border block in Fig. 9 relates to a distinctive feature of the present 
invention. 

[0089] Figs. 1 0 and 1 1 are flowcharts showing a conventional example of the macroblock layer multiplexing operation, 
35 and "®" and "® " in Fig. 10 respectively indicate that the relevant lines continue to "(3)" and "@" in Fig. 1 1 . 

[0090] Figs. 1 2 and 1 3 are flowcharts showing a (first) example of the macroblock layer multiplexing operation accord- 
ing to the present invention, explained with reference to List 4 and Fig. 6, and "(3)" and "®" in Fig. 12 respectively indi- 
cate that the relevant lines continue to "®" and "®" in Fig. 13. 

[0091] Figs. 14 and 15 are flowcharts showing a (second) example of the macroblock layer multiplexing operation 
40 according to the present invention, explained with reference to List 5 and Fig. 7, and "(3)" and "®" in Fig. 14 respec- 
tively indicate that the relevant lines continue to "(3)" and "(2)" in Fig. 15. 

[0092] Fig. 1 6 is a flowchart corresponding to the whole predictive decoding program of video data. The demultiplex- 
ing and decoding process in the double-border block of this figure relates to a distinctive feature of the present inven- 
tion. 

45 [0093] Figs. 17 and 18 are flowcharts showing a conventional example of the macroblock layer demultiplexing and 
decoding operation, and "(3)" and "®" in Fig. 1 7 respectively indicate that the relevant lines continue to "(3) " and "® " 
in Fig. 18. 

[0094] Figs. 19 and 20 are flowcharts showing an example of the demultiplexing and decoding operation for a mac- 
roblock layer multiplexed using the above first method, and "(3)" and "®" in Fig. 19 respectively indicate that the rele- 
50 vant lines continue to "(3)" and "@" in Fig. 20. 

[0095] Figs. 21 and 22 are flowcharts showing an example of the demultiplexing and decoding operation for a mac- 
roblock layer multiplexed using the above second method, and "®" and "®" in Fig. 21 respectively indicate that the 
relevant lines continue to "(3)" and "®" in Fig. 22. 

[0096] As respectively shown in Figs. 19, 20, 21 , and 22, a suitable decoding process is determined according to the 
55 data structure which was determined when the data were encoded. 
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Claims 

1. A predictive encoding method of video data, in whicli one of a global motion-compensating process for predicting 
a global motion of the whole frame and a local motion-compensating process for predicting a local motion of each 

5 block in a frame Is selectively performed, wherein: 

if a current block to be processed was interframe-encoded, then a code word for indicating the prediction mode 

is inserted in an encoded data sequence of the current block, the code word indicating which of the global and 
local motion-compensating processes was used for predicting the current block, and the code word inserted 
10 after another code word indicating the encoding mode of the current block; 

othenA^ise. the code word for indicating the prediction mode is not inserted in the data sequence. 

2. A predictive encoding method as claimed in claim 1, wherein when the current block is block-skipped, the global 
motion-compensating process is always chosen and in the skipped block, the code word for indicating the predic- 

15 tion mode is omitted. 

3. A decoding method for decoding a data sequence encoded by the predictive encoding method as claimed in claim 
1, wherein: 

20 if the current block was interframe-encoded, then the code word for indicating the prediction mode is extracted 

from the data sequence and decoding is performed using the indicated prediction method; 
othenwise the code word for indicating the prediction mode is not extracted. 

4. A decoding method for decoding a data sequence encoded by the predictive encoding method as claimed in claim 
25 2, wherein: 

when the current block has been block-skipped, the code word for indicating the prediction mode is not 
extracted and a decoding process corresponding to the global motion-compensating process is performed. 

30 5. A storage medium storing a program for making a computer execute a method as claimed in any one of claims 1 
to 4. 

6. A storage medium storing data encoded by a method as claimed in any one of claims 1 and 2. 
35 Amended claims under Art. 19.1 PCT 

1 . A predictive encoding method of video data, in which one of a global motion-compensating process for predicting 
a global motion of the whole frame and a local motion-compensating process for predicting a local motion of each 
block in a frame is selectively performed, wherein: 

40 

if a current block to be processed was interframe-encoded, then a code word for indicating the prediction mode 
is inserted in an encoded data sequence of the current block, the code word indicating which of the global and 
local motion-compensating processes was used for predicting the current block, and the code word inserted 
after another code word indicating the encoding mode of the current block; 
45 othenwise, the code word for indicating the prediction mode is not inserted in the data sequence. 

2. A predictive encoding method as claimed in claim 1 , wherein when the current block is block-skipped, the global 
motion-compensating process is always chosen and in the skipped block, the code word for indicating the predic- 
tion mode is omitted. 

50 

3. A decoding method for decoding a data sequence encoded by the predictive encoding method as claimed in 
claim 1, wherein: 

if the current block was interframe-encoded, then the code word for indicating the prediction mode is extracted 
55 from the data sequence and decoding is performed using the indicated prediction method; 

othenwise the code word for indicating the prediction mode is not extracted. 

4. (Amended) A decoding method for decoding a data sequence encoded by the predictive encoding method as 
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claimed in claim 2, wherein: 

when the current block has been block-skipped, the code word for indicating the prediction mode is not 
extracted and a decoding process corresponding to the global motion-compensating process is performed. 

5 

5. (Amended) A storage medium storing a program for making a computer execute a method as claimed in any one 
of claims 1 to 4. 7 and 8. 

6. (Amended) A storage medium storing data encoded by a method as claimed in any one of claims 1 , 2, and 7. 

10 

7. (Added) A predictive encoding method as claimed in claim 1 , wherein when the current block is block-skipped, 
the code word for indicating the prediction mode is inserted in the encoded data sequence, after a code word for 
indicating presence/absence of block skipping of the current block. 

15 8. (Added) A decoding method for decoding a data sequence encoded by the predictive encoding method as 
claimed in claim 7, wherein: 

when the current block has been block-skipped, the code word for indicating the prediction mode is extracted 
from the encoded data sequence, and a decoding process is performed according to the indicated prediction 
20 mode. 
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